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PREFACE 


Meteorologists  have  long  been  aware  of  the  problem  of  providing  meteo¬ 
rological  information  in  a  meaningful  form  for  use  by  decisionmakers. 

Some  decisionmakers  will  accept  no  meteorological  advice  while  others 
insist  on  a  "yes"  or  "no"  determination  regarding  any  question  of  the 
future  state  of  the  atmosphere,  and  a  rare  few  will  allow  range  meteoro¬ 
logists  to  present  their  forecasts  in  terms  of  probabilities.  Obviously 
there  will  always  be  a  degree  of  uncertainty  in  weather  forecasts,  therefore, 
it  is  imperative  that  decisionmakers  be  educated  to  accept  weather  forecasts 
which  are  provided  to  assist  in  the  decision  process  in  terms  or  probabilities. 
In  actuality,  most,  if  not  all,  inputs  used  in  arriving  at  a  decision  are 
uncertain.  If  range  meteorologists  are  forced  to  give  "yes"  or  "no"  meteoro¬ 
logical  inputs,  then  they  are  denied  the  option  of  expressing  the  degree  of 
uncertainty.  A  single  probability  number  may  contain  all  the  operator 
needs  to  know  about  the  weather,  its  effects  on  a  particular  aspect  of  the 
mission  objective,  and  the  confidence  of  the  meteorologists  in  his  forecast 
given  a  particular  situation  (Lyon  and  Leblanc,  1976).  If  the  probability 
forecast  is  used  as  an  input  into  an  operational  decision  model,  the  useful¬ 
ness  should  be  even  more  impressive.” 

Since  1970,  more  and  more  defense  agencies  have  learned  to  depend  on  oper¬ 
ational  decision  models  for  assistance  in  making  day-to-day  decisions. 

Yet  range  meteorologists  have  been  unsuccessful,  perhaps  reluctant,  to 
make  weather  probabilities  a  part  of  the  decisionmaking  process.  This 
document  presents  a  methodology  that  will  permit  more  complete  and  effec¬ 
tive  use  of  probabilistic  planning  and  operational  weather  forecasts  at  the 
ranges.  If  implemented,  this  support  information  will  provide  considerable 
cost  savings  and  increased  scheduling  efficiency  to  both  users  and  range 
operators. 

This  document  provides  range  meteorologists  with  an  introduction  to  the 
basic  concepts  and  operations  of  decisionmaking.  It  Is  not  expected  to 
make  the  reader  an  expert.  Instead,  the  material  will  provide  an  awareness 
of  the  capabilities  of  operational  decision  models  and  typical  situations 
where  they  should  be  used,  and  demonstrate  to  range  customers  the  value  of 
using  meteorological  forecasts  in  decisionmaking. 
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1.0  INTRODUCTION 


1.1  General.  The  job  of  every  range  meteorologist  Is  to  provide  the  best 
meteorological  support  possible  to  customers.  This  report  is  an  attempt  to 
improve  the  capabilities  of  those  providing  meteorological  advice  for 
decision  assistance. 

a.  Chapter  2  provides  guidance  on  how  to  apply  probabilistic  weather 
forecasts  in  decisionmaking.  The  basic  models  are  described  along  with  some 
simple  examples  to  explain  the  use  of  such  models. 

b.  The  contents  of  Chapter  3,  which  is  the  heart  of  this  report, 
should  grow  with  time.  It  contains  proven  cases  of  meteorological  assis¬ 
tance  in  decisionmaking.  In  some  of  the  cases,  customers  did  not  accept  the 
advice  eventhough  the  techniques  were  proven.  In  others,  the  customers 
gratefully  acknowledged  the  support  of  their  range  meteorologists  and  have 
documented  the  cost  savings  resulting  from  that  support.  Chapter  3  will  be 
expanded  as  more  and  more  use  the  guidance  in  Chapter  2  (and  other  sources) 
and  document  their  successes. 

1.2  Background.  It  is  assumed  users  of  this  document  have  a  knowledge  of 
statistics  and  can  prepare  probabilistic  weather  forecasts.  The  reference 
material  listed  in  the  bibliography  will  provide  a  source  of  reference  for 
those  who  need  further  training  or  retraining.  The  text  book  by  Panofsky 
and  Brier  and  Air  Weather  Service  Technical  Report  (AWSTR)  77-267  are  very 
good  sources  for  basic  statistics  review.  Chapter  6  of  Air  Weather  Service 
Publication  (AWSP)  105-51  contains  a  helpful  review,  of  the  forms  of  prob¬ 
ability  forecasts,  i.e..  Weather  Impact  Indicators  (WII),  and  how  WII  can 
be  used  to  predict  the  success  of  a  particular  mission  (Mission  Success 
Indicators  (MSI)).  A  more  detailed  discussion  of  customer  support  using  MSI 
is  given  in  the  referenced  Air  Weather  Service  Research  Paper  (AWSRP)  105-2. 

1.3  Terms/Definitions.  Basic  terms  and  definitions  are  updated  in  Appen¬ 
dix  AT  It  fs  recommended  that  readers  review  them  now  and  use  them  as  they 
work  with  this  report.  Some  terms  are  included  which  are  not  mentioned  in 
this  report  but  used  in  several  of  the  publications  referenced. 


2.0  PROBABILITY  FORECASTING  IN  RANGE  SUPPORT  DECISIONS 

2.1  Introduction.  The  range  meteorologist  can  aid  the  Range  Commander  in 
effecting  tremendous  savings  through  the  proper  use  of  probabilistic  fore¬ 
casts  when  making  weather-sensitive  decisions.  Since  weather  has  a  bearing 
on  almost  every  facet  of  testing  those  systems  designed  to  operate  in  the 
natural  environment,  the  opportunity  exists  for  a  wide  range  of  applications 
of  probability  forecasts.  By  the  introduction  of  this  kind  of  forecast  and 
indoctrination  of  range  personnel  in  Its  proper  use,  the  range  meteorolo¬ 
gist  can  make  a  substantial  contribution  to  his/her  range's  cost  avoidance 
program.  The  national  and  military  weather  services  are  moving  toward  the 
increased  use  of  probability  forecasts,  and  all  weather  services  are  faced 
with  similar  problems  of  educating  the  customer.  Range  meteorologists,  in 
particular,  seldom  find  recipients  of  this  kind  of  forecast  well  versed  in 
the  use  of  this  information.  The  meteorologist  thus  has  an  Inherent 
responsibility  to  furnish  the  guidance  needed  to  use  his/her  forecasts 
most  effectively  (Glahn,  1964;  Lyon  and  Leblanc,  1976).  The  objective  of 
this  guidance  should  not  be  to  make  the  customer  an  expert  in  decision 
theory,  which  is  a  complete  field  of  study  in  itself.  This  chapter  provides 
most  of  what  you  need  to  present  some  of  the  simpler  techniques  which  can 

be  applied  to  weather-related  decision  problems.  Specifically,  it  describes 
a  general  decision  matrix,  illustrates  applications  of  the  simple  cost-loss 
model,  defines  critical  probability,  and  demonstrates  methods  for  calculating 
the  value  of  forecast  information.  Brief  examples  illustrate  the  concepts 
introduced  in  this  chapter. 

2.2  General  Decision  Matrix.  Decisionmaking  under  risk  considers  that  one 
of  two  or  more  future  events  may  occur,  each  with  a  specified  probability. 
This  can  apply  to  meteorological  situations  in  which  the  frequencies  of  the 
various  future  weather  states  are  estimated  or  predicted,  i.e.,  probability 
forecasts  (Epstein,  1962).  A  matrix  is  the  most  convenient  method  for 
summarizing  all  the  elements  Involved  in  weather  decision  problems.  The 
generalized  form  of  a  decision  matrix  using  expenses  as  a  measure  of  value 
is  shown  in  table  2-1.  It  can  be  used  directly  or  may  serve  as  the  frame¬ 
work  for  developing  specialized  models. 


( 


2-1-1 


Table  2-1.  General  Expense  Matrix 
(Murphy,  1976b) 
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STATES  OF  WEATHER 
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a.  Explanation. 

(1)  In  the  general  matrix,  all  possible  courses  of  action,  stra¬ 
tegies,  or  decision  options  under  consideration  are  listed  in  the  left 
column,  i.e.,  aj  .  ..am  ...  a^  (m  =  1,2,...,M).  Under  the  states  of  weather, 
the  notations  Wj  ...Wn  . . .W^  ...(n  =  1,2,...,N)  represent  the  various  weather 
thresholds  which  have  an  effect  on  one  or  more  of  the  courses  of  action.  For 
each  action-state  pair  (am,Wn)  there  is  a  corresponding  consequence  or  out- 
come  (emn)  which  represents  the  expense  that  results  for  action  am  if  weather 
state  Wn  occurs  (Murphy,  1966). 
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(2)  Each  weather  probability,  P  ,  represents  the  probability 
that  the  weather  state  Wn  will  occur.  The  sum  of  all  the  weather  probabil¬ 
ities  must  equal  one  (Pj+  ...  +Pn  +  ...  +Pn  =1). 

(3)  Given  the  expense  associated  with  each  action-state  pair  and  the 
probabilities  of  each  state  of  weather,  the  long-term  expected  expense  (E)  can 
be  calculated  by  using  the  equations  in  the  right-hand  column.  For  example, 
the  expense  E  resulting  from  action  aj  would  be  computed  as  follows: 

Ei  =  Pi^n  +  ...  +pn6in+  •••  +PNe1ir  (2-1) 


Assuming  the  decisionmaker  wants  to  minimize  his  losses,  the  course  of 
action  selected  is  the  one  which  yields  the  smallest  value  for  Em  (Murphy, 

1976b),  i.e.,  the  course  of  action  which  will  cost  the  decisionmaker  the 
least  amount  over  the  long-term,  provided  the  probabilities  are  reliable. 

b.  Example.  Consider  the  situation  in  which  a  range  commander  must 
decide  between  four  forms  of  protective  action  for  his  aircraft  when  faced 
with  the  threat  of  a  hurricane. 

(1)  Table  2-2  Is  a  matrix  showing  how  this  decision  problem  would 
be  set  up.  Four  wind  thresholds  are  listed  under  the  states  of  weather.  As 
structured,  this  type  of  model  might  help  decide  what  action  should  be  taken 
regardless  of  what  is  causing  the  threat  of  wind  damage.  Assuming  the  range 
commander  wants  to  minimize  expected  costs,  the  costs  associated  with  each 
consequence  ( emn)  must  be  obtained  and  entered  in  the  matrix.  There  may  be 

many,  but  for  this  example  we  will  consider  only  two  types  of  costs. 


Table  2-2.  Incomplete  Cost  Matrix  for 
Protection  Against  Wind  Damage 


ACTIONS 

STATES  OF  WEATHER 

EXPECTED 
COSTS  (E) 

Wi  •  WIND 

<30  kts 

W2  =  WIND 

>30<50  kts 

W3  =  WIND 

>50<65  kts 

W4  =  WIND 

>65  kts 

=  No  Protection 

a2  =  Tie  Down 

a 3  =  Hangar 

a4  =  Evacuate 

PROBABILITY 

pi  - 

P2  * 

P3  « 

P4  - 

(a)  First  is  the  cost  of  taking  each  of  the  actions  indicated. 
Assume  that  the  figures  given  in  table  2-3  reflect  the  costs  obtained  from 
the  customer.  They  would  include  such  factors  as  manpower  required  to  tie 
down,  hangar  and  unhangar  aircraft;  costs  for  evacuation;  the  flying  costs 
to  and  from  the  refuge  base  with  TOY  expenses;  and  nonroutine  costs  generated 
by  the  action  taken. 


Table  2-3.  Costs  of  Taking  Protection  Action 
(Thousands  of  Dollars) 
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(b)  The  other  costs  would  be  the  estimated  cost  or  losses  as 
a  result  of  damage  when  aircraft  are  not  protected  or  when  protection  is 
inadequate.  Table  2-4  will  be  used  to  represent  these  costs. 


Table  2-4.  Potential  Losses  Due  to  Wind  Damage 
(Thousands  of  Dollars) 


ACTION 

Mi 

w2 

w3 

W4 

al 

$0 

$300 
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$12,000 
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0 
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These  figures  would  also  be  supplied  by  the  customer.*  As  these  values 
reflect,  the  potential  loss  varies  with  the  degree  of  protective  action 
taken  and  the  severity  of  wind  thresholds. 

(c)  To  obtain  the  total  costs  or  expenses  associated  with  each 
consequence  (e  )  of  the  decision  matrix,  the  corresponding  values  in 

tables  2-3  and  2-4  would  be  added.  Table  2-5  shows  the  resultant  matrix 
whose  data  are  now  ready  to  be  applied  to  a  decision  problem. 

Table  2-5.  Cost  Matrix  for  Protection  Against  Wind  Damage  Prior  to  Use 

(Thousands  of  Dollars) 


ACTION 

STATES  OF  WEATHER 

mmummm 

■ KK 
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(d)  Assume  a  hurricane  is  threatening  the  installation  (a 
possibility  at  two- thirds  of  the  RCC  ranges)  and  that  the  forecast  prob¬ 
abilities  for  the  different  states  at  time  of  landfall,  24  hours  from  now, 
are  as  follows:  P(WX)  =  5%,  P(W2)  =  80%,  P(W3)  =  10%,  and  P(W4)  =  5%. 

Expected  costs  (Em)  would  be  computed  as  shown  below: 


Em  =  piemi  +  P2e, 


m2 


+  p3em3  +  p-e. 


m1* 


(2.2) 


*AS5F  178-2  provides  guidance  in  computing  cost  figures.  Standard  cost 
factors  are  included  in  AFR  173-10,  Vol  I.  Similar  publications  are  avail¬ 
able  from  other  sources. 


I 
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E !  =  (.05  X  0.0)  + 
E2  =  (.05  X  1.0)  + 
E3  =  (.05  X  4.0)  + 
E4  =  (.05  X  120)  + 


(0.8  X  300)  +  (0.1 
(0.8  X  1.0)  +  (0.1 
(0.8  X  4.0)  +  (0.1 
(0.8  X  120)  +  (0.1 


X  1,500)  +  (.05 
X  601)  *  (.05  X 
X  4.0)  +  (.05  X 
X  120)  +  (.05  X 


X  12,000)  =  $990 
6,001)  =  $361 
1,504)  =  $79 
120)  =  $120 


By  entering  the  probabilities  and  expected  costs  in  their  appropriate  matrix 
positions,  the  final  decision  matrix  as  illustrated  in  table  2-6  can  be 
obtained. 


Table  2-6.  Final  Cost  Matrix  for  Protection  Against  Wind  Damage 

(Thousands  of  Dollars) 


STATES  OF  WEATHER 

| 

ACTIONS 

Wi 

<30  kts 

w2 

>30  A  <50  kts 

W3 

>50  &  <65  kts 

W4 

>65  kts 

EXPECTED 
COSTS  (E) 

ax  =  No  Protection 

$  o 

$300 

$1,500 

$12,000 

$990 

a2  =  Tie  Down 

1 

1 

601 

6,001 

361 

a 3  =  Hangar 

4 

4 

4 

1,504 

79 

a4  =  Evacuate 

120 

120 

120 

120 

120 

PROBABILITY 

Px  =  .05 

warn 

P3  =  0.1 

IBS 

(e)  The  decision  rule  assumed  earlier  is  that  the  preferred 
choice  is  the  course  of  action  which  results  in  the  least  expected  cost. 

By  using  this  rule,  course  of  action  a3  (hangar  the  aircraft)  is  preferred 
for  this  set  of  probabilities.  Different  combinations  of  probabilities 
would  yield  different  values  of  expected  costs  and,  thus,  different  deci¬ 
sions.  However,  when  one  course  of  action  affords  total  protection,  such 
as  evacuate  (a4),  the  expected  cost  (E)  of  that  action  will  remain  unchanged. 

(2)  Two  key  assumptions  in  this  decision  process  are  that  the  proba- 
bilities  are  reliable  and  the  expected  costs  are  long-term  averages.  The 
effect  of  the  latter  assumption  can  be  illustrated  by  one  of  the  computa¬ 
tions  for  expected  costs.  Computation  of  Ei  shown  under  equation  2.2  above 
is  repeated  for  Illustration: 


Ei  =  (-05  X  0.0)  +  (0.8  X  300)  +  (0.1  X  1,500)  +  (.05  X  12,000)  =  $990. 
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The  first  component  of  E]  contributes  nothing  to  the  expected  cost  because 
there  was  no  potential  loss;  i.e.,  no  damage  will  occur  as  long  as  the 
winds  are  less  than  30  knots.  In  the  second  component,  the  0.8  means 
that  8  times  out  of  10  (reliable  forecasts  assumed)  the  wind  will  be  with¬ 
in  that  threshold  (30  kts  <  V  £  kts).  On  each  of  those  eight  occasions 
the  damage  will  amount  to  $300K  with  no  damage  on  the  other  2  days  (total 
=  $2400K).  The  average  damage  amount  is  $2400K  divided  by  10  occasions  or 
$240K  which  is  0.8  x  300.  Similar  reasoning  applies  to  the  remaining  com¬ 
ponents.  Thus,  if  the  forecasts  are  totally  reliable  (bias  =  0),  average 
costs  will  equal  the  expected  costs  in  the  long-term.  Otherwise,  actual 
costs  will  differ  in  proportion  to  the  net  reliability  error  (bias). 

2.3  Utilities.  Money  (dollar  value)  is  the  most  cottmon  unit  of  value 
used  to  represent  consequences  of  decision  actions  (emn).  However,  it  is 
very  difficult  to  assign  a  monetary  value  to  many  types  of  consequences 
such  as  loss  of  military  readiness,  political  impact,  loss  of  prestige, 
loss  of  human  life,  and  reduced  combat  effectiveness  in  a  conflict.  In 
order  to  account  for  monetary  value  as  well  as  nonmonetary  factors,  the 
term  "utility"  was  coined  as  the  unit  of  value  of  consequences  when  non¬ 
monetary  factors  are  involved.  Utility  is  an  all  encompassing  term  which 
reflects  a  decisionmaker's  true  value  (preference  or  importance  weight) 
associated  with  a  given  consequence  or  outcome  (Murphy,  1976b). 

a.  Utility  Matrices.  A  utility  matrix  takes  the  same  form  as  that 
of  the  general  expense  matrix  (table  2-1).  The  only  difference  is  sub¬ 
stitution  of  utility  value  (umn)  for  expenses  (e  )  for  each  consequence 
and  expected  utility  (U)  for  expected  expense  (E).  Utility  values  can 
be  either  positive  or  negative.  The  objective  is  to  maximize  positive 
utilities,  such  as  profits  or  economic  gains,  and  to  minimize  negative 
utilities. 

b.  Transformation  of  an  Expense  Matrix  into  an  Equivalent  Utility 
Matrix.  In  general,  if  a  customer's  utilities  are  linearly  related  to 
the  respective  expenses  of  the  consequences,  an  expense  matrix  can  be 
transformed  directly  into  an  equivalent  utility  matrix  with  an  arbitrary 
scale  ranging  from  0  to  1 .  The  equation  for  performing  the  transformation 
(Murphy,  1976a)  is  given  by: 

“mr.  *  -  eL)/<eH  ‘  eL>  (2-3) 

where: 


umn  =  the  value  equivalent  to  expense  emn  (ranges  0  to  1). 

emn  =  the  exPense  value  of  the  consequence  being  transformed, 
e^  =  expense  value  of  the  least  preferred  consequence, 
e^  =  expense  value  of  the  most  preferred  consequence. 
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By  using  this  transformation,  the  most  preferred  consequence,  e^  (the 
one  of  least  expense)  takes  on  the  utility  value,  umn  =  1  (the  greatest 
utility).  Likewise,  the  least  preferred  consequence,  e^  (the  largest 
expense),  transforms  to  the  utility  value,  umn  =  0  (the  least  utility). 

(1)  Example.  Table  2-7a  is  an  abbreviated  form  of  the  general 
expense  matrix  shown  earlier  (table  2-1).  We  will  use  this  table  to 
demonstrate  the  transformation  technique  described  above. 


i 


i 


Table  2-7a.  Abbreviated  Expense  Matrix 


rt> 

*— • 

t— ' 

II 

1 

cn 

o 

ei2  =  70 

el  3  =  85 

e2i  =  5.0 

622  s  50 

e23  =  90 

e31  15 

o 

cn 

n 

CM 

CO 

0) 

e33  =  95 

From  table  2-7a  we  find  that  the  most  preferred  consequence  (eM)  is  ex, 
and  the  least  desired  (eL)  is  e33.  In  this  example  then,  equation  2.3 
takes  the  form: 

u  =  (e  -  95)/(-5. 0-95)  =  (emn  -  95)/-100  (2.4) 

mn  mn  '  mn  '  '  ' 

Substituting  values  for  e  ,  we  obtain  the  equivalent  utility  values  u 
shown  in  table  2-7b. 


Table  2-7b.  Abbreviated  Equivalent  Utility  Matrix 


C 

»—* 

II 

o 

u32  s  *25 

Uj  3  =  .10 

u2i  ='0.9 

U22  s  *45 

u23  =  .05 

u31  =  0.8 

u32  =  .65 

u  3  3  =  .00 
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(2)  Such  a  transformation  is  useful  for  two  reasons.  First,  it 
assigns  the  highest  utility  value  (1)  to  the  most  preferred  consequence 
and  places  the  decision  objective  of  maximizing  utilities  in  a  positive 
sense.  Second,  it  establishes  a  standard  scale  from  0  to  1  to  which  the 
customer  can  better  relate  by  using  ratios  to  confirm  whether  or  not  the 
equivalent  utilities  do  in  fact  reflect  true  preferences.  If  adjustments 
to  the  utilities  are  required,  this  scale  simplifies  and  expedites  the 
modifications. 

2.4  General  Cost-Loss  Model.  The  literature  on  probability  forecasting 
frequently  makes  reference  to  the  "cost-loss"  model.  The  cost-loss  model 
is  a  very  simple  and  specialized  case  of  the  general  decision  model  given 
earlier.  It  provides  a  realistic  description  of  situations  faced  by 
many  decisionmakers  and  has  been  used  extensively  by  meteorologists  and 
others  in  the  civilian  community.  This  model  was  originally  developed 
to  describe  a  situation  where  the  decisionmaker  must  decide  whether  or 
not  to  take  protective  action  with  respect  to  some  activity  or  operation 
based  on  an  uncertain  forecast  of  adverse  weather.  However,  it  also  has 
other  applications  when  only  two  courses  of  action  are  under  consideration 
(protect  or  not  protect,  go  or  no  go).  The  basic  cost-loss  model  assumes 
that  protective  action  completely  eliminates  losses  due  to  adverse  weather. 
However,  in  many  situations  all  resources  cannot  be  protected;  in  others, 
the  protective  actions  available  to  the  decisionmaker  only  reduce  the 
losses.  A  version  of  the  general  cost-loss  model  which  accounts  for 
unprotected  losses  is  shown  in  table  2-8  (Murphy,  1976a). 


Table  2-0.  Matrix  for  the  General  Cost-Loss  Model 


ACTION 

STATES  OF  WEATHER 

EXPECTED 
EXPENSES  (E) 

ADVERSE 

NOT 

ADVERSE 

ax  =  Protect 

C  +  L1 

C 

Ej  =  C  +  PiL' 

a2  =  No  Protection 

L  +  L' 

0 

E2  =  Pi (L  +  L') 

PROBABILITY 

_ ] 

P2 

a.  Terms.  In  this  model  the  cost  of  protection  is  denoted  by  C, 
losses  which  are  unprotectable  by  L',  and  the  protectable  losses  by  L. 
No  loss  or  cost  results  when  no  protection  is  taken  and  no  adverse 
weather  occurs. 

b.  Explanation.  Expected  costs  are  calculated  like  the  general 
expense  matrix,  Pj  =  C  +  Px 1  *  and  P2  =  Pj(L  +  L'),  since  Pj  +  P2  =  1. 
Now  assume  the  decisionmakers  want  to  select  protection  which  minimizes 
expected  costs.  We  equate  the  two  expected  costs  (Lx  and  L2)  and  solve 
for  the  probability  associated  with  taking  action  Pj.  Thus,  when  Pj  = 
C/L  (the  cost-loss  ratio),  the  expected  costs  are  equal.  By  the  same 


♦ 

A 


token,  if  Pi  >  C/L,  the  expected  cost  is  least  for  aj  (protect).  On  the 
other’hand,  if  Pj  <  C/L,  action  a2  (no  protection)  yields  the  least 
expected  cost.  Note  that  the  unprotectable  losses  do  not  enter  into  the 
decision  logic.  For  example,  if  this  model  was  applied  to  a  wind  damage 
decision,  such  unprotectable  items  (L1)  as  buildings,  towers,  fences, 
power  lines,  etc.,  would  not  be  included  unless  they  are  afforded  protec¬ 
tion  by  the  action  taken.  To  make  economic  sense,  the  ratio  C/L  must 
have  a  total  range  between  zero  and  unity.  If  C/L  is  greater  than 
unity,  the  cost  of  protection  would  exceed  the  loss  and,  therefore,  be 
uneconomical.  Similarly,  negative  values  of  C/L  are  also  economically 
meaningless  (Thompson  and  Brier,  1955). 

c.  Example.  Assume  the  test  engineer  has  some  wind  sensitive 
targets  set  up  for  an  extended  period  of  air-to-ground  tests.  If  winds 
speeds  greater  than  25  miles  per  hour  occur,  the  targets  will  be  destroyed 
at  a  cost  (loss)  of  $5000.  However,  the  targets  can  be  removed  and  stored 
in  a  hangar  at  a  cost  of  $500  for  labor  and  materials.  The  most  economical 
course  of  action  can  be  determined  by  computing  the  cost/loss  ratio  (C/L) 
and  comparing  it  to  the  probability  of  windspeeds  greater  than  25  miles  per 
hour.  For  this  situation,  C/L  =  500/5000  =0.1.  Therefore,  the  targets 
should  be  removed  if  the  probability  of  winds  greater  than  25  miles  per 
hour  is  more  than  10  percent. 

2.5  Critical  Probability.  In  the  discussion  of  cost-loss  models,  a 
decision  rule  was  derived  by  which  the  cost-loss  ratio  determined  the 
probability  threshold  above  which  protective  action  should  be  taken. 
Critical  probability  is  an  extension  of  the  cost-loss  ratio  concept  in 
that  it  can  be  applied  to  any  two-by-two,  action-state  decision  matrix. 

a.  Derivation.  Critical  probability  (Pc)  may  be  derived  by  using 
the  procedures  of  the  cost-loss  ratio  and  given  the  consequences  A,  B, 

C,  and  D  (in  utility  units)  from  tables  2-9a  and  2.9b  below,  while 
remembering  that  Pj  +  P2  =  1. 


Table  2-9a.  Protection  Matrix  for  Definition 
of  Critical  Probability 


ACTION 

STATES  OF  WEATHER 

EXPECTED  UTILITIES 

_  JU) _ 

Storm/ 

Rain 

No  Storm/ 
Rain 

at  =  Protect 

A 

C 

Ui  =  PiA  +  P2C 

a2  =  No  Protection 

B 

D 

U2  3  PiB  +  P2D 

PROBABILITY 

Pi 

P2 

I 
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Table  2-9b.  Launch  Matrix  for  Definition 
of  Critical  Probability 


ACTION 

STATES  OF  WEATHER 

EXPECTED  EXPENSES 
(U) 

Favorable 

Unfavorable 

a  i  =  Go 

A 

C 

Ui  =  PjA  +  P2C 

a2  -  No  Go 

B 

D 

U2  =  PiB  +  P2D 

PROBABILITY 

Pi 

P2 

p  -  ^  ~  ^ 

c  B  +  C  -  A  -  D 

The  decision  rule  for  a  critical  probability  is: 
Act  (a  )  if  Pj  >  Pc 
Indifferent  (aj  or  a2)  if  P!  =  Pc 
No  Action  (a2)  if  Pj  <  Pc 


(2 


(1)  Critical  probability  is  the  threshold  or  breakeven  probabil¬ 
ity  above  which  it  is  cost  effective  for  a  decisionmaker  to  take  a  specific 
action,  i.e.,  the  long-term  positive  utility  (value,  payoff,  etc.)  max¬ 
imized  and  the  negative  utility  (cost,  loss,  expense,  regret,  etc.)  is 
minimized.  It  may  be  based  on  monetary  value  or  other  measures  of  utility. 
Note  that  the  critical  probability  must  be  stated  in  terms  of  the  weather 
event  which  causes  the  action  to  be  taken.  This  is  a  subtle  but  Important 
point  and  the  reason  two  different  examples  are  given.  In  the  first  matrix 
action  is  taken  when  unfavorable  or  adverse  weather  (storm,  rain,  etc.) 
threatens.  But  in  the  second  case,  the  action  is  associated  with  favorable 
weather. 


(2)  Equation  2.5  reduced  to  P  =  C/L  for  the  general  cost-loss 
model .  c 


b.  Matrix  Example.  Consider  a  test  flying  operation  as  depicted  in 
the  matrix  of  table  2-10.  This  is  a  range  mission  in  support  of  a  cus¬ 
tomer  who  is  supplying  some  TOY  test  personnel. 


2-1-11 


Table  2-10.  Test  Flying  Matrix  (Dollars) 


ACTION 

STATES  OF  WEATHER 

EXPECTED  EXPENSES  (E) 

Favorable 

Unfavorable 

ai  =  Fly 

Ei  =  1000Pi-5700P2 

a2  =  Standdown 

B  =  -  1200 

D  =  -1200 

E2  =  1200Pi-1200P2 

PROBABILITY 

Pi 

Pz 

(1)  Definitions. 

A  is  the  benefit  realized  by  the  customer  when  the  weather  is 
favorable  and  the  mission  goes.  In  this  case,  the  benefit  less  operating 
costs  was  $1000. 

B  is  the  cost  (negative  benefit)  incurred  if  the  test  stands 
down  and  the  weather  is  favorable  --  a  lost  opportunity.  Assume  a  missed 
testing  day  costs  $1200  in  additional  TOY  funds  for  the  test  personnel. 

C  is  the  cost  or  loss  if  the  customer  takes  action,  but 
because  of  unfavorable  weather  cannot  accomplish  the  mission  (aborts). 

Each  test  mission  is  a  1.25-hr  flight  by  an  F-4.  Assuming  the  mission  is 
aborted  because  of  unfavorable  weather  over  the  test  site,  the  costs  would 
be  $4500  (1.25  hrs  X  3600/hr)  plus  $1200  for  another  TDY  day  (total  $5700). 

D  is  a  cost  or  benefit.  If  there  is  a  cost  for  mission 
delay,  it  is  a  cost.  If  a  delay  has  no  cost,  the  abort  cost  can  be  saved 
and  D  is  in  effect  a  cost  avoidance  benefit  (correct  standdown).  In  this 
example,  the  customer  considers  this  a  delay  cost  of  $1200. 

Pi  is  the  probability  that  no  weather  factors  (ceiling, 
visibility,  wind,  hazards,  etc.)  will  cause  mission  cancellation,  abort,  or 
failure  from  launch  to  recovery.  This  is  called  a  tailored  probability 
forecast.  Recall  that  Pi  +  P2  =  1  and  therefore,  P2  =  1 -P i . 

(2)  Explanation.  By  using  equation  2.5,  the  critical  probability 
for  this  example  is: 


p  -  -5700+1200 

c  "  T2ffd'-57M-100o-12OO  “  ,b/ 
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Thus,  the  decision  rule  for  this  decision  problem  is: 


Fly  if  Pi  >  .67 
Indifferent  if  Pi  =  .67 
Standdown  if  Pi  <  .67 


Referring  to  the  matrix  in  table  2-10,  the  expected  expense  for  each  mission 
will  equal  $1200  when  the  probability  of  favorable  weather  is  Pi  <  .67,  and 
gets  smaller  as  Pi  increases. 

(3)  Transformation  to  utilities.  In  the  example  above,  the  criti¬ 
cal  probability  of  67  percent  will  result  in  a  significant  number  of  missed 
opportunities.  Suppose  the  supported  agencies  complained  about  the  recent 
number  of  cancellations  due  to  weather  and  stated  that  it  is  essential  for 
their  test  project  to  complete  12  missions  during  the  next  20  days.  Faced 
with  this  situation,  the  range  meteorologist  agrees  to  help  work  out  a  com¬ 
promise  in  the  critical  probability  used  for  making  their  launch  decisions. 

(a)  Applying  the  utility  transformation  equation  (2.3)  to 
the  expense  matrix  v table  2-10),  the  meteorologist  prepared  an  equivalent 
utility  matrix  (table  2-11)  and  showed  it  to  the  test  agency.  Test  agency 
personnel  were  appalled  by  the  importance  weight  indicated  by  the  utility 
value  (B  =  .67)  for  a  standdown  with  favorable  weather  (missed  opportunity). 
They  were  satisfied  with  the  most  preferred  (A=l)  and  least  preferred  (C=0) 
consequences,  but  the  other  two  did  not  reflect  their  true  preferences  in 
the  present  situation.  After  discussion  between  the  range  meteorologist 
and  personnel  from  the  supported  agency  and  range  test  operations,  conse¬ 
quence  B  was  adjusted  to  a  value  of  1.0  because  this  consequence  was  now 
considered  nearly  as  undesirable  as  the  least  preferred  consequence.  This 
action  should  significantly  reduce  the  number  of  missed  opportunities  and 
satisfy  the  supported  agency.  Consequence  D  was  also  adjusted  to  a  lower 
value  (0.6). 


Table  2-11.  Test  Flying  Equivalent  Utility  Matrix 


ACTIONS 

STATES  OF  WEATHER 

EXPECTED 
UTILITIES  (U) 

Favorable 

Unfavorable 

ax  *  Fly 

A  =  1.0 

C  =  0.0 

Ui  -  Pi 

a 2  =  Standdown 

B  =  .67 

D  =  .67 

U2  =  .67 

PROBABILITY 

Pi 

-  .  .  .  _ 

P2 
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(b)  With  these  adjustments  in  utilities  (table  2-12),  a 
modified  critical  probability  was  calculated  as  follows: 


V 

> 


n  _  0.0-0. 6 

pc  '  o.i+'oTff^r^.'g  • 

Therefore,  the  new  decision  rule  becomes: 

Fly  if  P]  >  0.4 
Indifferent  if  Pi  =  0.4 
Standdown  if  P]  <  0.4 

Range  test  operations  personnel  state  that  they  are  much  more  comfortable 
with  this  decision  rule  because  it  reduces  the  number  of  lost  opportunities 
and  should  help  satisfy  the  needs  of  the  supported  agency. 


Table  2-12.  Modified  Test  Flying  Utility  Matrix 


ACTIONS 

STATES  OF  WEATHER 

. . . 

EXPECTED 
UTILITIES  (U) 

Favorable 

Unfavorable 

_ 

ax  =  Fly 

A  =  1.0 

warn 

Ui  =  Pi 

a2  =  Standdown 

B  =  0.1 

D  =  0.6 

PROBABILITY 

Pi 

P2 

c.  Other  factors.  Many  other  factors  can  be  incorporated  into  the 
test  flying  matrix.  For  example,  the  customer  may  not  be  another  DoD 
agency,  but  a  contractor.  In  such  cases,  TDY  costs  can  be  stated  in  terms 
of  relief  of  the  contractor  from  penalties  if  the  contract  is  not  completed 
in  time  due  to  unfavorable  weather.  Still  another  matrix  can  be  devised, 
one  not  restricted  to  flying  costs.  This  might  add  the  expense  of  opera¬ 
ting  theodolites  and  radars,  range  crews,  and  data  reduction  requirements 
to  aircraft  costs. 
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d.  Merits  of  Using  Critical  Probabilities. 

(1)  The  obvious  advantage  of  using  critical  probabilities  in 
decisionmaking  is  that  they  can  be  predetermined  by  the  decisionmaker 
and  appropriate  action  implemented  whenever  the  critical  probability 
threshold  is  exceeded.  Thus,  some  decisions  can  be  made  without  direct 
involvement  by  the  decisionmaker. 

(2)  The  use  of  monetary  value  is  a  good  starting  point  for 
determining  critical  probability.  When  actual  values  are  not  available, 
rough  approximations  are  usually  adequate.  The  accuracy  of  the  critical 
probability  need  not  be  any  more  than  one-half  the  value  of  the  probability 
intervals  used  in  making  the  forecasts.  That  is,  if  forecasts  are  made  up 
0-19,  20-39,  40-59,  60-79,  80-100,  the  critical  probability  needs  to  be 
correct  to  within  10  units  of  the  actual  critical  probability. 

(3)  Critical  probabilities  can  be  adjusted  either  objectively 
or  subjectively  as  priorities  and  other  factors  that  affect  the  decision 
change.  For  example,  a  test  director  may  establish  a  critical  probability 
for  use  when  test  missions  are  on  schedule,  but  if  testing  falls  behind 
schedule,  a  lower  value  (depending  upon  the  number  of  missions  needed  and 
time  remaining)  can  be  substituted. 

(4)  If  a  customer  is  opposed  to  using  probability  forecasts 
directly,  critical  probabilities  furnish  an  alternate,  but  formal,  way 
of  providing  tailored  categorical  forecasts.  Rather  than  using  50  per¬ 
cent  as  the  threshold  for  deciding  whether  or  not  an  event  will  occur, 
the  critical  probability  may  serve  as  the  threshold.  In  the  long  run, 
the  resultant  decisions  will  be  more  cost-effective  than  conventional 
categorical  forecasts. 

e.  Problems  in  Using  Critical  Probabilities. 

(1)  A  customer's  critical  probability  may  be  outside  the  limits 
within  which  reliable  forecasts  can  be  reasonably  assured.  In  such  a 
case,  the  customer  should  be  making  decisions  based  on  climatology. 

(2)  Forecasters  may  let  the  value  of  the  critical  probabilities 
influence  the  value  of  their  forecast  probabilities.  Beware  of  this  fix¬ 
ation  because  there  may  be  occasions  when  a  customer  changes  the  critical 
probability  without  the  forecaster's  knowledge. 
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3.0  EXAMPLES 


Specific  applications  of  probability  forecasting  and  decision  assistance 
vary  from  range  to  range.  Examples  of  either  or  both  should  be  made 
available  to  those  at  the  other  ranges  so  that  they  may  profit  by  the 
knowledge  and  success  experienced  by  each  other.  While  every  example 
may  not  be  directly  applicable  to  your  range,  it  may  trigger  an  idea 
on  how  to  do  the  job  better.  This  chapter  is  expected  to  grow  rapidly 
with  your  help. 
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3.1  ARMAMENT  DIVISION  (AD) 

3.1.1  Situation 


a.  The  AD  (formerly  the  Armament  Development  and  Test  Center)  has  a 
centralized  scheduling  Directorate  office  which  coordinates  all  test  and 
training  missions  for  the  Division.  This  Test  Engineering  Directorate  (TED) 
does  not  consider  weather  forecasts  in  its  original  request  for  missions  over 
the  Eglin  AFB  land  and  water  range  complex.  The  schedule  is  printed  several 
days  prior  to  the  valid  period  which  precludes  accurate  weather  forecasts  as 
input  data.  At  noon  on  the  day  prior  to  tests  (rundown),  the  schedule  is 
finalized.  Some  flexibility  exists  at  this  point  which  can  allow  missions 
to  be  cancelled  or  added,  or  objectives  changed. 

b.  The  engineers  responsible  for  tests  coordinate  at  their  discre¬ 
tion  with  the  duty  forecaster  as  their  respective  test  period  approaches 
but  rarely  cancel  based  on  forecasts.  As  of  this  writing,  the  AD  charges 
range  users  only  if  they  use  the  range.  This  encourages  engineers  to  wait 
until  the  last  minute  before  cancelling  or  changing  test  objectives. 

Since  the  range  is  vast  and  many  resources  are  committed  several  hours  in 
advance,  most  last-minute  changes  must  be  minor. 

c.  The  AD  meteorologists  are  convinced  that  test  scheduling  and 
mission  success  rates  can  be  improved  by  coordinating  weather  forecasts 
along  with  other  parameters  in  the  scheduling  process  at  rundown  time. 

This  would  allow  more  flexibility  to  cancel  and  add  missions  or  to  better 
coordinate  test  objective  changes.  To  this  end,  the  meteorologists  began 
in  March  1977  to  issue  planning  forecasts  (PF's)  every  morning.  Each  PF 
was  valid  for  the  following  day  from  0600  to  0000L.  The  hard-copy  fore¬ 
casts  contained  normal  forecast  parameters  in  standard  coding  and  addi¬ 
tionally  provided  probabilities  of  occurrence  for  ceilings  greater  than 
1,500,  3,000  and  10,000  feet;  visibility  greater  than  1,  3  and  7  statute 
miles;  no  rain;  no  TSTM;  no  fog;  and  surface  wind  less  than  25  knots  (see 
figure  3-1-1).  These  forecasts  were  sent  to  TED  to  assist  in  the  daily  test 
scheduling  process. 

3.1.2  Results 

a.  Although  the  PF  was  not  used  in  the  scheduling  process  as  Intended, 
it  became  widely  accepted.  By  July  1977  the  TED  was  reproducing  17  copies 
which  were  distributed  to  Test  Engineering  offices  and  posted  in  sections 
for  use  by  test  engineers.  These  forecasts  alerted  engineers  to  weather 
factors  which  might  affect  scheduled  tests.  Further  reproductions  of  the 
forecasts  were  common,  although  the  distribution/reproduction  chain  was 
not  traced  further. 

b.  An  attempt  (unsuccessful)  was  made  to  prove  to  the  Test  Wing  that 
the  weather  forecasts  could  save  It  money.  The  4-month  period  December 
through  March  1978  was  chosen  for  the  exercise,  and  only  periods  when  the 
weather  was  observed  or  forecast  to  be  less  than  12,000/6  (12,000-foot 
ceiling  and  <  6  miles  visibility)  were  looked  at.  This  narrowed  the  scope 
of  the  analysis  to  days  when  the  weather  could  affect  test  missions. 
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Each  test  mission  scheduled  during  this  period  was  monitored  to  determine 
if  the  forecast  (based  on  the  test  engineers'  requirements)  was  either 
good  or  bad.  Those  requirements  changed  from  test  to  test. 

c.  Table  3-1-1  summarizes  the  248  test  missions  evaluated. 


Table  3-1-1.  Test  Mission  Summary 


FORECAST 

OBSERVED 

GOOD 

BAD 

GOOD 

127 

48 

BAD 

16 

57 

(1)  The  weather  for  127  missions  was  correctly  forecast  when  the 
tests  were  conducted  successfully  or  could  have  been  conducted  successfully 
during  days  when  the  weather  was  forecast  or  observed  to  have  conditions 
less  than  12,000/6.  It  is  significant  to  note  that  this  number  would  be 
much  larger  if  clear  weather  days  had  been  Included  in  the  analysis. 

Knowledge  of  good  weather  conditions  correctly  forecast  is  important,  but 
was  excluded  to  make  this  analysis  tougher  on  the  meteorologist. 

(2)  The  weather  for  57  missions  was  forecast  correctly  when  the 
missions  were  not  conducted  due  to  adverse  weather.  All  of  these  missions 
could  have  been  cancelled  at  1200  the  day  prior  to  the  scheduled  test  period 
or  preparations  begun  as  a  result  of  changes  in  mission  objectives.  This 
figure  is  probably  low  because  test  engineers  used  the  PF  to  change  some 
mission  objectives  and  conducted  successful  missions  that  would  otherwise 
have  been  cancelled. 

(3)  The  weather  for  48  missions  was  forecast  to  be  good  when  in 
fact  it  was  too  poor  to  conduct  the  tests.  This  figure  represents  aborts 
for  the  crews  and  missed  opportunities  for  the  meteorologists  to  provide 
accurate  forecasts. 

(4)  The  weather  for  16  missions  was  forecast  bad  and  the  observed 
weather  was  good.  If  tests  had  been  cancelled  because  of  the  PF,  these 

16  missions  would  have  been  lost  opportunities.  Table  3-1-1  figures  look 
bad  for  the  met  team  until  the  costs  for  these  248  missions,  based  on 
the  weather  forecasts  versus  the  "fly-them-all"  attitude  that  existed 
before,  are  considered. 

3.1.3  Value  analysis 

a.  Proponents  of  the  "fly-them-all"  attitude  say:  "if  you  can  take 
off  'go,'  try  the  mission  because  the  forecaster  might  be  wrong  and  you  can 
get  a  successful  mission."  As  can  be’  seen  from  table  3-1-1,  that  is  true  for 
16  cases  but  very  false  for  57  cases. 


3-1-3 


b.  To  compare  values,  the  costs  for  missions,  set  up,  per  diem,  etc., 
were  needed.  These  costs  were  not  available  since  the  missions  were  dif¬ 
ferent  and  the  test  times  variable.  However,  the  meteorologists  and  the 
test  engineers  came  to  a  temporary  agreement  on  the  data  in  table  3-1-2. 

It  was  assumed  that  there  was  no  value  to  a  forecast  that  allowed  the  mis¬ 
sion  to  proceed,  since  that  was  the  standard  philosophy  --  "fly-them-al 1 . " 
There  is  room  for  argument  about  that  assumption,  although  it  is  of  no 
consequence  here.  In  addition,  it  was  presumed  that  if  the  crew  flew  a 
mission  based  on  a  forecast  of  good  weather  and  had  to  abort,  at  least 
$10,000  would  be  lost.  Also,  if  a  forecast  of  bad  weather  caused  a  mission 
cancellation  24  hours  in  advance,  and  the  weather  was  good,  at  least  $2000 
would  be  required  for  per  diem,  rescheduling,  etc.  Lastly,  if  the  forecast 
of  bad  weather  caused  a  mission  cancellation,  and  the  weather  was  indeed 
bad,  $5,000  could  be  saved  in  fuel,  repairs,  wasted  preparations,  etc. 


Table  3-1-2.  Observed  States  of  Weather 


ACTION 

i 

GOOD 

i 

BAD 

FLY 

0 

-$10,000 

STAND  DOWN 

-$2,000 

$  5,000 

c.  Based  on  these  figures  and  those  in  table  3-1-1,  the  Division  incurred 
no  cost  (additional)  for  the  143  (127+16)  good  weather  periods.  Conversely, 
the  105  missions  during  the  bad  weather  periods  (48+57)  cost  $10,000  each, 
with  no  usable  data,  for  a  total  cost  of  $1,050,000. 

d.  Using  the  cost  figures  and  assuming  the  missions  were  flown  based 
only  on  the  weather  forecasts  (table  3-1-1),  $480,000  was  expended  for  48 
missions  that  had  to  abort;  $32,000  for  16  lost  opportunities;  and  $285,000 
was  saved  on  57  missions  that  were  properly  cancelled  because  bad  weather 
was  correctly  forecasted.  This  represents  a  net  cost  of  $227,000. 

e.  Although  the  actual  savings  that  could  have  resulted  are  still 
unknown,  a  potential  savings  of  $823,000  ($1,050,000  minus  $227,000)  was 
enough  evidence  to  convince  the  test  engineers  to  take  a  harder  look  at 
the  PF  provided  one  day  in  advance  of  each  mission.  A  year  later  there 
are  still  very  few  missions  cancelled  24  hours  in  advance,  more  test 
objectives  are  changed  (fly  the  high  profile  tomorrow  because  low  clouds 
are  forecast,  etc.),  and  many  more  requests  for  tailored  forecasts  are 
received.  AD  meteorologists  hope  to  refine  the  PF  and  eventually  provide 
tailored  forecasts  for  each  test  through  the  use  of  utility  matrices  and 
critical  probabilities. 
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3.2  EASTERN  TEST  RANGE  (ETR) 
3.2.1  Introduction 


As  part  of  the  support  provided  to  the  Eastern  Test  Range,  Detach¬ 
ment  II  (Det  II)  has  been  forecasting  launch  conditions  for  the  Navy's 
Trident  missile  development  program.  The  most  critical  constraint  is 
acoustical  propagation,  a  safety  factor  based  on  the  damaging  over¬ 
pressures  which  might  occur  should  a  missile  explode  immediately  after 
liftoff.  Temperatures,  lapse  rates  and  component  winds  are  the  important 
factors  in  determining  the  rate  and  direction  of  sound  propagation  and 
the  likelihood  that  "blast  focusing"  could  result  in  damage  to  nearby 
civilian  property. 

When  the  Trident  launches  began  in  January  1977,  Det  II  personnel 
briefed  their  customers  on  the  expected  conditions  and  provided  a  go/ 
no-go  forecast.  This  satisfied  Navy  personnel  because  they  assumed  Det  II 
met  people  were  completely  confident  of  their  forecasts,  but  they  were  not 
pleased  if  any  of  their  forecasts  were  less  than  precisely  accurate. 

To  improve  service,  Det  II  switched  from  go/no-go  forecasts  to  a 
probabilistic  format  beginning  with  the  Trident  launch  on  12  April  1978. 

It  quickly  developed  that  the  Navy  would  try  to  launch  if  provided  go 
conditions  higher  than  10  percent.  Although  Det  II  made  the  change  to 
numbers,  customers  continued  to  interpret  the  numbers  in  a  go/no-go  fashion. 
This  mode  still  did  not  allow  the  flexibility  to  communicate;  an  important 
benefit  of  probabilistic  forecasting. 

With  the  Trident  launch  on  11  August  1978,  Det  II  inaugurated  a 
different  form  of  probabilistic  forecasting.  The  percentages  the  met 
people  used  were  those  that  communicated  true  confidence  in  their  fore¬ 
casts.  For  the  first  four  or  five  briefings  Det  II  personnel  had  to  stop 
and  explain,  in  very  basic  language,  exactly  what  the  new  numbers  meant 
since  they  were  generally  much  higher  than  the  numbers  previously  used. 

A  20  percent  chance  of  go  conditions  would  be  described  as  a  go  condition 
during  the  the  launch  period  on  only  1  of  5  days  with  similar  synoptic 
conditions. 

The  Navy  and  ETR  decisionmakers  reacted  well  to  the  new  forecast 
format.  After  the  first  few  briefings  they  began  to  seriously  apply  the 
weather  inputs  Det  II  provided.  On  a  day  with  no  other  demands  on  Range 
use,  they  would  elect  to  attempt  to  launch  on  a  low  probability  forecast. 
While  on  other  days,  with  the  same  forecast  but  when  other  Range  users  were 
available,  they  would  not  attempt  to  launch. 

This  attempt  at  probabilistic  forecasting  was  limited  because  the 
customer  made  no  inputs  -  it  was  purely  a  forecaster-derived  number 
reflecting  his/her  confidence  in  the  forecast.  What  was  missing  was 
an  input  from  the  customer  describing  In  cost  or  utility  terms  the  effect 
of  favorable  or  unfavorable  weather  qn  the  decision  to  count  or  hold. 

Without  customer  inputs,  no  critical  probability  could  be  determined  above 
which  the  Navy  would  launch  or  below  which  the  Navy  would  hold.  The  per¬ 
centages  Det  II  met  personnel  had  been  providing  lacked  consistency  between 
forecasters  and  were  still  very  subjective. 
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3.2.2  Probability  Matrix 


Although  the  land-launched  Trident  program  is  now  over,  a  probability 
matrix  could  have  been  used.  The  analysis  follows: 

a.  Two  cost  factors  were  used  from  the  3  September  1977  value  analysis: 
Cost  of  a  Trident  countdown  =  $81,000;  Overhead  cost  for  keeping  the  range 

up  but  not  fully  manned  for  a  countdown  =  $19,000. 

b.  The  Launch  Matrix  for  Definition  of  Critical  Probability  is  contained 
in  table  3-2-1 . 


Table  3-2-1.  Launch  Matrix  for  Definition 
on  Critical  Probability 


ACTIONS 

OBSERVED 

STATES  OF  WEATHER 

EXPECTED 
EXPENSES  (E) 

Favorable 

Unfavorable 

Count 

A  =  81 

C  =  162 

Ej  =  APj  +  CP2 

Don’t  Count 

B  =  100 

D  =  100 

E2  =  DP]  +  DP2 

Pi 

P2 

A  =  Cost  of  counting  down  a  missile  and  launching  it  (most  desirable 
occurrence 

B  =  Overhead  cost  of  having  range  ready  and  not  counting  for  1  day, 

plus  the  cost  of  a  full  count  for  launching  the  next  day  (represents 
missed  opportunity) 

C  =  Cost  of  a  full  count  with  no  launch,  plus  the  cost  of  a  full  count  for 
launching  the  next  day  (least  desirable  occurrence) 

D  =  Basically  same  as  B;  Overhead,  plus  cost  of  count  during  launch  day 
(represents  area  of  greatest  potential  savings) 

P  =  Probability  a  given  weather  condition  will  occur 

c.  The  key  number  needed  is  the  Critical  Probability,  Pc.  This 
represents  the  probability  threshold  at  which  a  count/don't  count  decision 
should  be  made.  In  this  matrix: 


„  _  C-D  _  162-100  _  62  _  ,, 

pc  ‘  8+C-O  ‘  166+1 62-81'-T(50  "  ST  ~  *77 
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1 
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This  means:  Count  if  Pc  >  .77 

Indifferent  if  Pc  =  .77 
Don't  Count  If  P£  <  .77 

d.  There  are  some  problems  with  this  key  number.  Not  counting  with 
a  value  near  0.7  would  not  be  acceptable  to  the  customer  because  if  an 
opportunity  is  missed  it  might  be  days  before  another  favorable  day.  To 
handle  this  problem,  translate  the  dollar  figures  into  terms  of  utility. 
Do  this  by  using  the  formula: 


\ 


I 


u 


Where:  u  =  utility 

e  =  expense  of  consequence  being  transformed. 
eL  =  expense  of  least  desirable  consequence. 

em  =  expense  of  most  desirable  consequence. 


( 


The  previous  matrix  had:  A  =  81  C  =  162 

B  =  100  D  =  100 

The  most  expensive  consequence,  C,  becomes  e^  and  the  least  expensive  con¬ 
sequence.  A,  becomes  e  .  By  letting  e  equal  81,  100,  162,  and  100,  the  new 
matrix  becomes:  ' 


A  =  1.0  C  =  0.0 
B  =  .77  D  *  .77 

With  this  matrix,  the  most  desired  consequence  has  a  utility  of  1.0  and 
the  least  desired  consequence,  a  utility  of  0.0.  The  others  lie  somewhere 
in  between. 

e.  if  .77  Is  too  high  for  Pc  as  far  as  missed  opportunities  go, 
reassign  a  new  utility  for  B  that  more  closely  reflects  the  customer’s 
thoughts.  For  Trident,  a  missed  opportunity  (B)  is  only  slightly  more 
desirable  than  counting  during  unfavorable  conditions  (C).  If,  In 
coordination  with  the  customer,  B  was  lowered  from  .77  to  0.1,  the  new 
critical  probability  would  become:  ' 
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—  N  '  r. 


-5" 


This,  as  it  turned  out,  was  more  aligned  with  Det  II  customers'  desires 
and  comprised  a  more  realistic  countdown  decision  parameter.  This  form  of 
probability  matrix  is  called  a  Modified  Equivalent  Utility  Matrix. 


V 

> 


f.  There  are  some  obvious  problems  in  applying  the  above  analysis 
directly  to  countdown  decisions.  The  cost  figures  would  be  significantly 
greater  when  holding  a  launch  for  extended  periods,  as  was  frequently  the 
case.  However,  the  principle  of  the  probability  matrix  remains  since  the 
utility  terms  can  be  modified  to  an  acceptable  compromise  value  to 
account  for  these  extended  holds. 


i 
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3.3  WESTERN  TEST  RANGE  (WTR) 
3.3.1  Introduction 


The  Western  Space  and  Missile  Center  (WSMC)  manages  the  Western  Test 
Range  (WTR)  which  extends  from  the  launch  site  at  Vandenberg  AFB,  CA,  to 
the  Indian  Ocean.  The  weather  is  extremely  important  when  R&D  ballistic 
missile  launches  are  planned  because  of  uprange,  midrange  and  downrange 
weather  constraints.  Activation  of  all  facilities  and  sensors  necessary 
to  support  such  a  complex  launch  must  begin  several  hours  before  scheduled 
launch  time.  If  the  operation  is  scrubbed  late  in  the  countdown,  thousands 
of  dollars  (in  some  cases  hundreds  of  thousands)  In  range  costs  are  expended 
with  no  payoff.  To  avoid  these  costly  "weather  scrubs,"  WSMC  began  using 
probability  forecasts  for  decisions  to  activate  the  range  and  continue  a 
countdown.  A  very  special  mission  required  met  personnel  to  Increase 
their  efforts  in  this  area  and  the  results  were  outstanding. 

3.3.2  Background 

The  Minuteman  Natural  Hazards  Program  Involved  six  launches  of  specially 
designed  re-entry  vehicles  from  Vandenberg  into  predetermined  and  precisely 
defined  weather  conditions  in  the  Kwajaleln  Impact  area.  A  prime  objective 
was  to  evaluate  RV  performance  and  to  relate  that  performance  to  the  mete¬ 
orological  conditions  encountered  by  the  RV's.  The  success  of  each  of  the 
six  flights  was  essential  for  accomplishment  of  the  overall  program  object- 
tive.  Since  each  test  could  only  be  conducted  during  the  occurrence  of 
certain  weather  conditions  in  the  impact  area,  test  planners  were  faced 
with  the  problem  of  how  to  determine  the  best  times  to  schedule  range 
support.  To  activate  the  ranges  and  begin  the  missile  countdown  at  random 
times  or  repeatedly  on  a  day-by-day  basis  would  have  resulted  In  an  enor¬ 
mous  expense  and  would  have  required  virtually  full-time  dedicated  support 
by  range  resources  that  were  required  to  support  other  programs.  Weather 
probability  forecasts  were  therefore  relied  upon  to  limit  activation  of 
WSMC  and  other  support  range  resources  to  those  times  when  there  was  a 
reasonable  likelihood  that  the  necessary  weather  conditions  would  occur. 

3.3.3  Weather  Severity  Index 

A  weather  team  composed  of  a  Minuteman  Special  Projects  Office  (SPO) 
representative,  the  WSMC  meteorologist,  and  other  technical  consultants 
prepared  probability  forecasts  and  go/no-go  recommendations  which  were 
briefed  to  key  test  personnel  at  Vandenberg  via  telephone  conference  at 
critical  decision  points.  This  usually  occurred  12  hours  before  the 
scheduled  launch  time.  Based  upon  the  weather  team's  recommendations, 
test  planners  would  either  activate  the  resources  of  all  ranges  involved 
In  the  test  and  begin  the  countdown,  or  reschedule  the  test  and  plan  to 
evaluate  the  situation  at  the  critical  decision  point  for  the  new  launch 
window.  Conventional  weather  forecasts  were  of  limited  value  for  those 
tests.  A  critical  parameter  upon  which  decisions  were  based  was  a  Weather 
Severity  Index  (WSI),  a  complicated  function  of  the  liquid  water  content. 

Ice  water  content,  and  Ice  crystal  structure  In  the  re-entry  corridor. 

This  number  could  be  directly  related  to  expected  re-entry  vehicle  per¬ 
formance.  The  WSI  was  extremely  variable  spatially  and  temporally  because 
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of  the  frequent  convective  shower  activity  in  the  area.  Probability  fore¬ 
casts  were  issued  based  on  the  predominant  cloud  features  expected,  the 
amount  and  duration  of  expected  convective  activity,  and  the  forecaster's 
confidence.  Conventional  meteorological  data,  plus  data  from  instrumented 
aircraft  and  special  radar  equipment,  were  used  to  verify  the  forecasts. 

3.3.4  Setting  Thresholds 


It  became  apparent  very  early  in  the  test  series  that  the  "bad"  weather 
requirements  (those  calling  for  large  MSI  values)  would  be  difficult  to 
satisfy.  In  order  not  to  miss  an  opportunity,  it  became  necessary  to 
"threshold"  at  a  very  low  probability  value.  For  the  high  WSI  launches,  go 
decisions  were  generally  made  if  the  probability  of  success  was  greater  than 
about  20  percent.  In  retrospect,  when  the  tests  were  successfully  concluded, 
it  was  determined  that  the  high  WSI  values  occurred  only .10  percent  of  the 
time,  so  the  20-percent  threshold-value  was  reasonable.  (This  is  an  impor¬ 
tant  concept  for  those  who  use  probability  forecasts  to  understand.  If  the 
climatological  expectancy  of  the  desired  weather  is  small  and  the  mission 
urgency  is  high,  thresholds  should  be  set  low  so  an  opportunity  will  not  be 
missed.  If,  on  the  other  hand,  the  weather  is  easy  to  obtain  and/or  the 
mission  urgency  is  not  great,  thresholds  should  be  set  high  so  that  resources 
are  not  wasted.  The  decisionmaker  should  set  these  threshold  values  in 
advance,  although  real-time  adjustments  will  sometimes  be  necessary.) 

3.3.5  Innovations 


Two  innovative  procedures  were  developed  as  a  result  of  WSMC's  involve¬ 
ment  with  the  Minuteman  Natural  Hazards  Program.  Since  the  high  WSI  require¬ 
ments  were  hard  to  satisfy  and  since  there  was  normally  a  large  spatial 
variation  in  WSI  values,  a  real-time  retargeting  capability  was  developed 
that  allowed  targeting  of  the  RV's  into  any  one  of  three  widely  separated 
impact  areas  as  late  as  20  minutes  before  launch.  Using  the  sampling  air¬ 
craft  and  radar,  the  weather  team  could  then  make  last-minute  recommenda¬ 
tions  as  to  which  target  was  to  be  used.  The  second  innovation  was 
implemented  to  use  the  range  resources  more  efficiently. 

Again,  since  high  WSI  values  ("bad"  weather)  were  hard  to  find  and  low 
WSI  values  ("good"  weather)  were  relatively  easy  to  find,  WSMC  adopted 
the  practice  of  scheduling  high  and  low  WSI  missions  for  the  same  time 
and  then  making  a  decision  based  on  the  probability  forecasts  made  6  to 
12  hours  prior  to  launch  as  to  which  mission  would  be  activated.  With 
this  procedure,  seven  "good"  weather  launches  were  completed  during  a 
period  when  "bad"  weather  launches  had  the  highest  priority.  Thus,  WSMC 
avoided  a  potentially  serious  scheduling  problem  that  could  have  adversely 
impacted  other  range  users'  programs. 

3.3.6  Results  Achieved 

Over  a  period  of  14  months,  6  Minuteman  Natural  Hazards  Program 
launches  were  successfully  completed  with  13  actual  countdowns,  7  of  which 
were  terminated  prior  to  launch  because  the  weather  criteria  could  not  be 
satisfied.  If  we  assume  that  without  special  weather  probability  forecasts 
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and  reasonably  established  threshold  values,  the  ranges  would  have  been 
activated  each  time  a  launch  was  scheduled  and  scrubbed  late  in  the  count 
if  criteria  could  not  be  met,  31  attempts  would  have  been  required.  A 
documented  value  analysis  has  shown  that  by  avoiding  18  unsuccessful 
countdowns  through  the  use  of  weather  probability  forecasts,  a  net  avoid¬ 
ance  of  $3,200,000  in  range  support  costs  was  achieved  for  this  program. 
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4.0  IMPLEMENTATION  AND  CAPABILITIES 

4.1  Deficiencies  and  Recommended  Improvement  Areas 

Deficiencies  in  Probabilistic  Meteorological  Decision  Assistance  occur 
in  two  primary  areas:  implementation  and  capabilities.  To  be  effective, 
the  implementation  of  probability  forecasts  must  proceed  through  four  phases: 
development,  tesing,  evaluation,  and  operational  use.  These  phases  take 
time,  direction  and  funding.  The  decision  to  allocate  resources  to  develop 
probability  forecast  capabilities  must  come  from  levels  high  enough  to  fund 
for  these  resources.  The  decisionmakers  must  believe  that  the  benefits  of 
increased  capability  are  worth  the  costs  of  the  capability.  Long-range 
accuracies  must  indeed  demonstrate  that  users  can  decrease  risk  of  loss. 
Studies  such  as  those  in  Chapter  2  of  this  document  serve  as  a  good  starting 
point. 

4.2  Meteorological  Probabilities 

Although  meteorological  probabilities  are  used  operationally  within  the 
National  Weather  Service,  they  are  not  generally  employed  within  the  DoD  and, 
in  particular  at  DoD  Test  and  Evaluation  Ranges.  For  them  to  be  employed 
would  require  training  programs  for  the  meteorological  personnel  involved. 
Such  programs  would  consider  the  weather  parameters  of  interest,  a  clima¬ 
tology  of  these  parameters,  the  impact  of  these  parameters  on  the  customer's 
mission,  and  the  techniques  for  assessing  the  probability  of  occurrence  of 
these  parameters.  The  customer  also  must  be  trained  to  understand  the 
meaning  of  the  event  forecast.  Most  of  the  current  misunderstandings 
of  probability  forecasting  on  the  part  of  the  public  stems  as  much  from 
the  uncertainty  of  the  meaning  of  the  event  (for  example,  precipitation  at 
a  point  as  opposed  to  precipitation  over  an  area)  as  it  does  from  the 
uncertainty  of  the  meaning  of  the  probability.  Furthermore,  the  customer 
must  understand  the  impact  of  various  weather  elements  on  the  operations 
in  question,  and  must  be  able  to  put  dollar  values  on  the  Impact  of  these 
weather  occurrences  in  order  to  assess  actions  to  be  taken  when  specific 
probabilities  are  forecast. 

4.3  Centralized  Facilities  Vs.  Field  Activities 

A  decision  must  be  made  on  whether  probability  forecasts  will  be 
developed  from  centralized  facilities,  in  the  field,  or  a  combination  of 
the  two.  Conceivably  probabilities  can  be  issued  for  large  areas  by  a 
centralized  facility  and  modified  by  field  forecasts  for  local  use.  A 
possible  Innovation  would  be  the  Incorporation  of  the  Air  Weather  Service's 
Output  Statistics  (MOS)  Into  range  probability  forecasts.  However,  MOS 
and  similar  data  need  to  be  tested  and  verified  for  accuracy  before 
routinely  accepted  and  used. 

4.4  Climatological  Data 

Finally,  there  are  deficiencies  In  climatological  data  and  other  base¬ 
line  information  which  could  be  applied  to  assess  the  operational  effective¬ 
ness  of  new  high-technology  systems.  Such  environmental  variables  as 
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infrared  and  microwave  transmissions,  exceptionally  long-range  visibi  lity, 
refractive  index,  stability  index,  and  optical  turbulence  fall  into  this 
category.  Information  on  these  elements  is  already  being  sought,  but  the 
availability  is  nearly  negligible  except  in  a  very  few  areas  convenient  to 
laboratories.  An  alternative  would  be  to  establish  correlations  among  these 
elements  and  the  standard  observables  (temperature,  humidity,  cloud  cover, 
visibility,  etc.)  for  which  extensive  climatologies  are  extant.  Unfortu¬ 
nately,  there  is  much  disagreement  Qn  the  potential  effectiveness  of  such  a 
program.  Again,  the  decision  to  establish  programs  such  as  these  must  be 
deferred  to  those  empowered  to  allocate  funds. 


o 
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APPENDIX  A 


TERMS  EXPLAINED 

1.  Probability.  The  chance  that  a  prescribed  event  will  occur,  represented 
as  a  number  ranging  from  0  to  1 .  The  probability  of  an  impossible  event  is 
0.0,  that  of  an  inevitable  event  is  1.0.  The  percentage  equivalent  (0  to 
100  percent)  is  frequently  substituted  when  discussing  probabilities;  how¬ 
ever,  the  decimal  equivalent  (0  to  1)  should  be  used  when  performing  mathe¬ 
matical  computations. 

2.  Climatological  Probability.  The  probability  that  an  event  will  occur 
based  on  extensive  historical  observations  or  experimental  data.  The 
simplest  form  of  climatological  probability  (commonly  called  climatic  fre¬ 
quency)  is  the  number  of  occurrences  of  an  event  divided  by  the  sum  of  the 
number  of  occurrences  and  nonoccurrences  over  a  given  time  period.  More 
complex  forms  of  climatological  probability  frequently  use  climatic  models 
when  historical  observations  are  not  available.  In  these  cases,  the  models 
are  used  to  obtain  estimated  climatological  probabilities  of  the  desired 
event. 

3.  Sample  Climatological  Probability.  The  climatological  probability 
based  on  observations  that  are  made  only  during  a  sample  period.  Examples 
are  climatological  probabilities  based  on  one  month's  data. 

4.  Objective  Probability.  The  probability  that  an  event  will  occur  based 
on  a  fixed  set  of  rules  which  produce  a  unique  and  reproducible  outcome. 

The  rules  may  be  derived  by  empirical  or  theoretical  considerations  or 

a  combination  of  both. 

5.  Subjective  Probability.  A  personal  estimate  of  the  probability  that 
an  event  will  occur.  Subjective  probability  estimates  give  good  results 
if  the  individual  knows  the  forecast  problem  (dynamics  of  the  situation, 
climatology  of  the  event,  etc.)  and  is  aware  of  basic  probability  laws 
and  limitations  of  forecast  skill.  Subjective  probability  forecasts  may 
not  be  reproducible. 

6.  Event.  A  specific  occurrence  that  is  defined  by  a  weather  element(s), 
time,  location,  and/or  duration;  e.g.,  visibility  less  than  1  mile  in  the 
period  1700-2000Z  lasting  more  than  30  minutes  at  Scott  AFB.  Some  events 
do  not  require  all  of  the  above  specifications,  e.g.,  rain  at  Offutt  AFB 
at  0600Z. 

7.  Probability  Forecast.  Meteorological  advice  consisting  of  two  parts— 
a  well  defined  weather  event  and  the  expectation  that  the  event  will  occur. 

8.  Post  Agreement.  A  measure  of  how  often  an  event  occurs  when  it  was 
forecast  (forecast  hits  divided  by  total  forecasts).  This  Is  a  measure 
of  categorical  forecasting  reliability. 
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9.  Prefigurance.  A  measure  of  how  often  an  event  was  forecast  when  it 
occurred  (forecast  hits  divided  by  total  occurrences).  This  is  a  measure 
of  categorical  forecasting  capability. 

10.  Correlation.  The  measure  of  how  well  the  forecasts  agree  with  the 
observed  weather.  Correlation  values  range  from  0  to  +1 ,  where  -1  is  per¬ 
fect  negative  correlation,  0  is  no  correlation,  and  +1  is  perfect  positive 
correlation  (Reference  AWS  TR  75-259). 

11.  Sharpness.  The  degree  of  certainty  of  a  probability  forecast.  A  set 
of  forecasts  containing  only  0  percent  and  100  percent  probability  values 
has  perfect  sharpness.  Zero  sharpness  occurs  if  all  forecasts  are  for  a 
probability  value  equal  to  the  sample  climatology. 

12.  Reliability.  The  degree  to  which  forecast  probabilities  resemble  the 
observed  frequency  for  each  forecast  probability  value  or  interval.  For 
example,  an  event  would  occur  80  percent  of  the  time  for  a  series  of 
perfectly  reliable  80  percent  probability  forecasts. 

13.  Decision  Theory.  A  set  of  rules  designed  to  use  probabilities  and 
other  information  to  make  an  optimal  decision:  information  about  the  state 
of  nature  (a  weather  forecast),  and  information  (utility,  value,  expense, 
regret,  etc.)  on  the  outcome  (consequence)  of  the  decision.  This  informa¬ 
tion  is  usually  given  in  the  form  of  a  utility  matrix. 

14.  Utility.  The  value  a  decisionmaker  associates  with  a  given  outcome 
with  respect  to  other  possible  outcomes.  It  may  be  based  on  monetary  value 
alone,  or  other  factors  which  influence  the  decisionmaker's  order  of  pre¬ 
ference  for  the  outcomes. 

15.  Utility  Matrix.  (Also  called  decision  matrix,  cost-loss  matrix,  expense 
matrix,  payoff  matrix,  value  matrix,  etc.,  depending  upon  the  writer  and  the 
way  outcomes  are  quantified).  A  two-dimensional  array  arranged  in  rows  and 
columns.  Normally,  rows  represent  possible  courses  of  actions  (strategies, 
options,  decisions)  and  columns  represent  the  different  states  of  nature 
(weather  categories  or  thresholds).  Entries  at  intersections  of  each  row 
and  column  represent  the  outcome  (utility,  cost,  loss,  expense,  payoff, 
value,  regret,  or  opportunity)  associated  with  each  course  of  action  and 
state  of  nature  pair. 

16.  Critical,  Threshold,  or  Breakeven  Probability.  The  probability  above 
which  it  is  cost  or  mission  effective  for  a  decisionmaker  to  take  a  speci¬ 
fic  action,  i.e.,  the  long-term  positive  utility  (value,  payoff,  etc.)  is 
maximized  and  the  negative  utility  (cost,  loss,  expense,  regret,  etc.)  is 
minimized.  Critical  probability  serves  as  the  threshold  which,  when 
exceeded,  generates  a  decision  to  act.  It  may  be  based  on  monetary 
value  or  other  measures  of  utility.  When  weather  is  the  only  factor 
affecting  the  decision,  the  critical  probability  must  be  stated  in  terms 
of  the  weather  event  which  will  cause  action  to  be  taken,  e.g.,  hangar 
aircraft  when  the  probability  of  hail,  exceeds  a  critical  probability  of 

10  percent.  When  other  variable,  nonweather  mission  factors  affect  the 
decision,  the  customer  may  use  a  critical  probability  stated  in  terms  of 
mission  success. 
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17.  Mission  Success  Indicator  (MSI).  The  probability  that  a  mission  will 
succeed.  An  MSI  is  tailored  to  a  specific  decision.  It  includes  both 
weather  (probability  forecasts)  and  nonweather  elements  that  are  needed 

to  make  an  optimal  decision. 

18.  Weather  Impact  Indicator  (WII).  A  WII  is  the  weather  input  for  deci¬ 
sion  assistance.  It  is  the  probability  of  e-eeding  a  particular  threshold 
of  a  given  weather  event  or  the  probability  distribution  of  the  weather 
event.  Customers  can  combine  the  WII  with  iv  «  ather  parameters  to  calcu¬ 
late  a  Mission  Success  Indicator  (MSI)  for  use  n  decisionmaking. 

19.  Climatological  Weather  Impact  Indicator  (CWII).  A  WII  based  on  cli¬ 
matological  probabilities  rather  than  forecasts.  CWII’s  are  useful  for 
planning  military  operations,  such  as  scheduling  events  or  selecting 
areas  or  routes. 

20.  Simulated  Weather  Impact  Indicator  (SWII).  AN  SWII  is  produced  by 
using  a  model  which  simulates  the  variability  of  observed  and  forecast 
weather  for  specified  climatic  regimes.  SWII's  can  be  used  independently 
(or  combined  with  nonweather  factors  to  produce  simulated  MSI's)  to  study 
the  impact  of  weather  and  weather  forecasts  on  operations,  for  training 
aids  and  illustrative  purposes,  or  to  assist  decisionmakers  in  the  optimal 
use  of  WII’s,  such  as  determining  critical  probability. 
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